COMMENT ⊗   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002
C00052 ENDMK
C⊗;

There are two kinds of talks.  I hope I don't disappoint you.  Fred asked
me quite nicely to talk.  Today he asked me what I was going to say.  I said
it would be more or less what I said to the freshmen seminar last year, and
I couldn't quite read the expression on his face.  There are two kinds of talks
about social effects.  One kind is the "Gee Whiz" talk about the wonders of the
future, and the other is the sort of gloomy thing where everything has terrible
side effects, you've got to be very careful, and so forth.  This is going to be
the former.

I just read an article about the social effects of computing, and it ended up
on the last page with a couple of statements that I want to project.  [Evident
difficulty with projection.]  I'll read it anyway.  The top is just a reference 
which  you can get later if you're serious.  What I'll quote is after a discus
sion of the various effects that if a company and a review of different 
studies....it says, "With such meager systematic attention it is hard to believe
that important understanding about the long-term or subtle social features of
computing ---------------------------."  I entirely agree with this.

The second is, "It is hard to believe that the problem could best be served by
rapid development of a poorly understood technology."  And there I entirely
disagree with him.  I think we should plunge ahead even if we don't understand
it.  It is pleasant to have that viewpoint since it goes with the viewpoint that
we will plunge ahead anyway, whether we understand it or not and whether the 
wise heads think we ought to.

I want to discuss the social effects of some systems that I think will be
shortly available, and I want to discuss them mainly in terms of opportunities
that they will provide to individuals and organizations. To regard this as
goods one has to take a certain sort of general view of humans and human society,
which is to say that people use such opportunities as they have for their own
benefit, and institutions in general use such opportunities as they have to carry  
out their normal functions and operations.  You sort of take a different view with 
regard to people of the future as our children, not in the obvious sense that 
they are our descendants, but in the sense of regarding them as children who are to be 
be protected from what they might otherwise mistakenly do.  Then you can take a
quite different view.  It is justifying the view that I will take.

I would say suppose that we should suddenly manage to find in some cornerstone of
the building some documents from 1880 saying what would be the dangers affecting 
the people of 1980, and what could the people of 1880 do to protect us from 
the mistakes that we might make?  I think that we would be very astonished if what
they said had any relevance to what we currently imagine our problems to be.

So what kind of thing am I discussing?  The cost of computer disk storage has now 
gone down to the point where it is just as cheap to store information on computer
disk files as it is to store it on a library shelf, even if you are talking about
one copy.  If you consider the cost of a book, cataloguing it, the cost of the
building, the furniture, and so forth.  There is no reason to suppose that this
will stop.  It will go down further.  Of course, one has to say that we do not
presently have the Library of Congress in computer files.  I would like to envisage
a system in which everything published is available on disk files and that anyone
who has one of his terminals at home and one in his office can read anything that
has been published by simply calling for it by card catalog number.  We can suppose
that for a long time such a system would co-exist with the present system of
publication, but one of its features would be that essentially publication in a
certain sense becomes immediate in that anyone who wishes to declare a file public
has in a certain sense published it.

Now what we know is that this isn't going to make people read much more than they
already do, so the problem of deciding what to read, or from the point of view of
the author of getting what he writes read will be just as acute as it is now,
maybe more so.  The current institutions which bring writing to attention would
be expected to exist in modified form, and it would be interesting to speculate on
what the modified form will be.  The first modification is that the production and
distribution function almost vanishes.  At present the cost of physically producing
books, distributing them, and making them available to bookstores comes to 80 or
90 percent of the cost of the book, at least judging from the author's royalties.
Our little --------.  And the balance could go the other way or at least could go
down to something like 50-50, even assuming that fairly substantial services were
done in finding errors and making the thing adhere to editorial standards and in
advertising of some kind.

One of the effects that we might expect is that many people who write would become
substantially independent of publishers.  Particularly someone who was famous
could have the reasonable assurance that his faithful readers would arrange to
be notified whenever something new by him appeared in the public file sytem.
Therefore, he could be independent of anyone who called himself a publisher, 
although he might not be quite so independent of people who call themselves book
reviewers.  They could still ignore him, as they can now.  One would expect from 
this a greater ability to serve small publics since the total number of readers
would be smaller and you wouldn't have the problem that because the public is
small the thing has limited availability.  One would expect more specialized
markets.  Probably scientists would be better satisfied.  For example, you
might get something that has its treatment of political news cater to us, or 
something like that.  One could even have journalists who were independent of 
publications.  In other words, you could imagine that there was somebody who was 
the equivalent of a correspondent in some foreign capital but who did not belong to
any publication as a regular thing because there were people who were interested
in the affairs of this country who would get his stuff regularly, which would mean
that for at least part of his public he wouldn't have to start from scratch in
every article.  One could expect something deeper from them.

Another effect which is interesting and might not be expected, at least it
took me a while to think of it, is that one could expect that there would be
a more effective right of reply.  It seems to me that one of the political
discoveries of the last 50 to 100 years has been the effectiveness of political
statements where a statement is made whose effect depends substantially upon
the fact that the person or institution that is attacked will not get a
reply to the same readers or hearers of the original statement, at least not
quickly. If we compare, as I did, the Lincoln-Douglas debates with the
Ford-Carter debate in 1976, one discovers that in the meantime it has been
discovered that there is no need whatsoever to be willing to answer a complicated
hypothetical question that your opponent has.  I think that if there was an
instant possibility of reply and what I mean by that is that anyone who reads
something would be able to ask the computer, "Well, has the target said anything
in reply to that?"  Then this would immediately appear on the screen.  You would
simply press that button.  Then the original person who made a statement would
have to make it in anticipation of the fact that there would be such a reply.
My hope is that, although certainly people who have hoped for social improvements
from new technology have been disappointed in the past - Armstrong who invented
--------- was very outspoken in his disappointment at what had become radio.  If
he could see what has become of TV, he would -----------------------------------.
I believe that these are two important social effects that one can hope for.

There are some other things that one might imagine if you work through computer 
terminals in your home.  And as a technical question one might say that there
has been a lot of development of home computers.  And of course you know that you
can buy something from Radio Shack for $600 which is as good a computer as
Whirlwind I was which occupied a whole building.  Now the unfortunate fact is
that the average person, and I must confess that in this respect I am an average
person, has relatively little use in his daily life for great ability to do
numerical calculations.  I have felt little urge to write basic programs.  I don't
want to decide what to buy or to keep track of the things in my pantry because,
first of all, inputting information would be too difficult.  So I think that the
home computer as a stand-alone object will remain a hobbyist affair.

However, what I believe almost everybody has a substantial use for 
is access to sources of information.  I have mentioned the Library of
Congress but including the equivalent of current publications as
important ----------.  But there are also other sorts of things like airline
schedules, or airline seating ability, the catalog of the local store which
if one saw that in such a condition as I believe---------more stores would find
it desirable to have catalog information available for the public.

There is also one dealing with bureaucracy which is to say if the government
sends you around this form to fill out, then one can ask them to make it
available through your computer, and certain parts of it you can essentially
say, "Yes, give it that information."  It will nicely transcribe your
address for the next - oh, I almost forgot my new civil right.  I once tried
to assert this civil right without very great success, and that is that no 
institution should be able to require a person to answer a question if it
already knows the answer.  I tried this on the Internal Revenue Service once,
but they simply uncomprehendingly sent the form back again.  My wife chickened
out.  I am not absolutely sure, you see, that the present law can in any way be
interpreted as requiring me to give that information again.  But in any case one 
can satisfy the bureaucrats more easily or for that matter defy them more easily.

Another somewhat more far-out thing has to do with negotiating programs. 
Namely, if there really are car dealers and the information about what they sell 
is available in the computer files, then you may use a program that someone will
be eager to sell you that will go negotiate with the car dealer, that is, will
negotiate with their programs and try to get you the best deal.  I don't know 
exactly what will become of that.

Now all of these things that I have mentioned are not only scientifically
available they are technologically available.  I believe that the obstacles 
are purely entrepreneurial.  In other words when some super-smart 
businessman succeeds in rousing the other businessmen to give a lot of money
or buy a lot of his stock, then most of these things are ready to be done.

Now I would like to discuss a few things which are not merely technologically
difficult but are even scientifically difficult.  They depend upon scientific
-------- that have not yet been solved.  Now I don't know how optimistic Herb
Simon expressed himself as being about the state of artificial intelligence.
I think we would entirely agree in our beliefs about the long-term future,
but I tend to be less optimistic than he has been about paths in evaluating
our -------- progress.  Of course, our journalists are always saying, "Well,
how long is it going to be until we have robots?"  I would like to say something
like this.  "Well, our distance from robots can be measured as 1.7 Einsteins,
and 5 Faradays, and .2 of the Manhattan Project, and it's important to have the
.2 of the Manhattan Project after the Einstein rather than before."  Anyway,
that is the answer to the question that you haven't even asked, and I will
procede.

I will mention a couple of more long-term things.  One would like to get a
greater degree of objectivity into human affairs - rationality.  Suppose we
merely say objectivity, the ability to take more facts into account, to be
more explicit about reasoning as to what facts have been taken into account.
On this I think I would like to mention some recent results in artificial
intelligence which bear on this problem.  Some of us, and I have to include
myself in this, had the following dream many years ago that if we could 
describe, express, common sense knowledge in sentences of mathematical logic,
and we could also describe particular situations then it should, in principle, be 
possible to logically deduce what should be done.  It turns out that if you 
sufficiently re-interpret this goal, it could still be maintained, but taken
in the sense at least in which I meant it twenty years ago, it turns out to be
not possible because it turns out that some non-rigorous reasoning methods are
required.  As soon as one produces the phrase "non-rigorous reasoning methods",
all sorts of people will leap in and say, "Yes, we know everything must 
inevitably be a total muddle and so forth and so on, but I don't mean that.
It turns out that some of these "non-rigorous reasoning methods", while 
non-rigorous, are perfectly formal and can be described in a way in which we
can make a computer carry them out.

The topic which has arisen and which is the subject of the latest issue of our
journal, ARTIFICIAL INTELLIGENCE, is what has been called non-monotonic
reasoning, non-monotonic logic.  The word, non-monotonic, comes in the
following way.  Ordinary mathematical reasoning is monotonic in the following
sense.  Suppose the conclusion P follows from a collection of sentences A.  If
you believe these sentences, all of the sentences in A, you will believe P.
And let us suppose that B is a more inclusive collection of sentences.  So
everything in A is a sentence of B and yet there are more sentences in B.  It
has this monotonistic property - if you increase the set of available premises,
then you may increase the set of possible conclusions.  And it turns out that 
ordinary human reasoning does not have this property, and if computers are to
reason successfully, we shall have to provide them with some means of reasoning 
that do not have this property.  There is a theorem of mathematical logic which
says, or --------- rather, which says that the methods of reasoning of mathematical
logic are complete in the sense that if you introduce new ones, you can arrive
at contradictions.  This is indeed a fact that one has to suffer with 
non-monotonic reasoning.  It isn't rigorous.

Let me give a couple of examples. One Russian -------- gave an example from
a Russian folk story.  A man says, "Well, I went to the market and bought
peas at a good price."  His interlocutor said, "Good." "But on the way home
they all fell out of the sack."  "Bad."  "However, they grew up along the
road to a nice crop of peas." "Good."  And you can continue the story.  It
surely is a classical Russian folk tale.  There are exactly three alternations.
The point is that as you add facts which do not contradict the previous facts, 
you change the conclusion.  What it amounts to being able to say is that what
we have to do to make computers go is something like taking the simplest model
of the facts available.  The computer must ask itself what conclusions can be
drawn in the simplest model of the facts available.  

Another example is the well-known missionary and cannibal problem.  Three
missionaries and three cannibals come to a river.  There's a boat that seats
two, and there's the unfortunate fact that the cannibals, while obedient,
are subject to temptation.  If the cannibals ever outnumber the missionaries
on either bank of the river, then the missionaries will be eaten.  How shall
they get across?  If you regard that story as racist or something like that, then
you can turn it around and you can say that if the missionaries ever outnumber
the cannibals on either bank of the river, then the missionaries will convert
the cannibals to their religion.  Now suppose you tell this to some uncooperative
dunce, and he thinks for a while and he says, "Why don't they cross on the
bridge?"  And you say, "There isn't any bridge."  And he says, "It doesn`t say
there isn't any bridge."  And the way it turns out in ---------- it`s a kind of 
rule of inference when you are given a description of a situation that you 
accept as conclusions those things that can be drawn from ----------- the simplest
model.  And the simplest model in this case could be given by Occam's razor,
namely, that entities whose existence does not follow from the statement of the
problem can be assumed not to exist.  

Actually, suppose that this isn't a puzzle and that you are seated by the bank
of the river and along come these people and they ask you this question.  You
would do two things.  One is, I claim, that you would do the same reasoning
as the puzzle and think about zigzagging back and forth across the river with
the boat as, a solution.  But after having done that, you would ask the question,
"Well, maybe there is a bridge, or maybe the river is fordable, or maybe if
you feed them up well beforehand, they won't be subject to temptation.  So
in real life this kind of non-monotonic reasoning is a rule of conjecture
rather than a rule of inference.  The puzzle can be regarded as a rule of
inference because you know there isn't going to be any more information.  But in
real life it is a kind of rule of conjecture.  These people working in 
artificial intelligence  are just beginning to be able to tackle the problems,
which is to say that we can discuss them theoretically, but as yet there are
few programs that do any substantial amount of non-monotonic reasoning.

Let me mention another puzzle of non-monotonic character.  This one is
considerably worse.  Imagine that there was a law making it a crime to attempt
to bribe a public official, and suppose that the legislators debated this 
law and passed it, and it was enforced for twenty years.  Then the following
thing came up.  Someone offered to -------------.  Well, it's true that I
offered him a thousand dollars to fix my drunk driving conviction, but I
didn't know he was a public official.  Now if you attempt to bribe this person,
who is in fact a public official, but not knowing it, then are you guilty of
attempting to bribe a public official?  Or you can have it the other way 
around which is you attempted to bribe this guy in the mistaken belief that he
was a public official and in fact he was not.  Or one that I just thought of
today, while talking to Fred.  You attempted to bribe this fellow whose 
commission as a public official had not yet been awarded.  He hadn't taken the 
job yet.  Let us imagine, to make matters more complicated, that in fact he dies
before he could take up the job.  Then did you attempt to bribe a public official?
Or an additional one is suppose you put an ad in the newspaper saying that I
will pay $l,000 to any public official who can fix my drunk driving conviction.
Can you attempt to bribe a public official without there being any specific
public official whom you are attempting to bribe?  

All of these problems, or problems very like them, have been dealt with by
philosophers, and many of them have been dealt with by lawyers as well.  The
philosphers call part of one of those distinctions the de re de dicto
distinction, which is made in the following way.  Suppose that John wants to
have an affair with Bill's wife.  How do we interpret that?  Ordinarily we
would feel that it was pretty easy to interpret but there are two possibilities
that come out.  One is that this woman that John wants to have an affair with
- but he has never heard of Bill - it's just that we know she is Bill's wife.
Is it required that he know?  That's the de re interpetation.  In that case, 
namely, the thing.  The thing is Bill's wife, but it isn't stated that he knows
anything about her being Bill's wife.  And the other is the de dicto interpretation.  
He wants to have an affair with Bill's wife because he's mad at Bill.  But, in
fact, Bill isn't even married.  So the philosphers know about that.

Now the interesting thing, the necessary thing, from the point of view of
artificial intelligence is not to solve these puzzles.  To go back to the point,
what did the legislators have in mind when they passed this law without ever
thinking of these puzzles?  Because we may be in a position that there are in
fact an indefinite number of such puzzles that can be invented.  The philosophers
have a slogan that you shouldn't have to do all science before you start on
philosophy.  It looks to us like we computer scientists need a slogan.  You 
shouldn't have to do all philosophy before you do A.I., artificial intelligence,
that is.  You shouldn't have to solve these puzzles in advance so that in some way
the formalism that is used should, at least at some level, be indifferent to the
de re de dicto distinction.  Then the computer program that has not made this
distinction should then, in some sense, be capable of being puzzled when a case
comes up that requires such distinction.  There is a philosopher, ------------
Dennett, who looked at this de re de dicto distinction and said, "Well, that
distinction isn't very clear.  We can introduce all sorts of additional cases
or complications to it, which aren't clearly the de re de dicto distinction, where
there are fifty distinctions.

This is the kind of thing to which we would like to apply some form of 
non--monotonic reasoning and would have to.  Namely, what we would like to be able
to say is that in some sense there is no ambiguity unless one arises.  In other
words, [laughter]    .  That was a digression that I went into because it was
something I felt like talking about, but its relevance to the question is that
I see true artificial intelligence as being a ways away because we do really
have to establish our attitude toward such questions.  We either have to do the 
philosophy or find out why we don't have to do the philosophy.

So let me regress a little bit and mention that with regard to this question
of using computers and formal reasoning within computers to increase
objectivity, which I think we can do, one of the things we will have to do is to
make it do some kind of non-monotonic reasoning.

This has relevancee to present computer model building.  It seems to me that
the limitations of present computer model building, and the use of computer
models and simulating systems, etc., has to do with the problem that you are
facing of what conclusions can be drawn from a given model, or are the 
issues one of what facts shall be taken into account?  When someone makes a
really big error, it is more likely that there are some important facts that
he didn't take into account.  Let me give a sort of hypothetical example.

Suppose some government, as governments have, had gotten sort of ecologically
minded, sort of against chemical fertilizers, and said, "Well, we should use
human waste to fertilize the fields."  It hired a company to do a study.
This company said we have good computer facilities, and we will take everything
into account and do a simulation.  They took into account the expense of
collecting it, how good fertilizer it would be, transmitting it to the fields,
and they had a time series of over the next twenty years for the introduction
of this thing, how much money it would save, etc.  But what if the real issue
turned out to be that this thing produced a disease path, produced another
path from the human digestive tract back into the mouth.  Then there is no
amount of detail in this computer simulation, taking the original things into 
account, which would make it correct.  It seems to me that in many problems where
people try to use computers that is their difficulty.

Now I want to do a little more, even further out, "Gee Whizzing."  Having said
I don't know how soon this will come about, let us consider the question of
robots.  I believe that eventually we shall have domestic robots that will
behave like servants.  I want to do a little bit of speculation about what that
would be like.  Of course, lots of people are worried about what would happen
when even machines less capable than robots can do all the work, and there is
no need for all these workers.  They paint gloomy scenarios.  I will paint my
scenario which is gloomy in a sense, maybe optimistic in a sense, but it has
its gloomy aspect, too.  I find that we already know what will happen because
it has happened in the past.  Namely, there has existed in human societies,
many societies, an upper class, and this upper class regarded
the lower class as a part of the machinery, and went on for hundreds of years
taking that view of things.  My suggestion is that when we want to discover
what it would be like if everybody was rich to look back at the societies in
which everybody who was anybody was rich. They got into lots of trouble.
They were very quarrelsome, I think is perhaps the main trouble, but one could
say that they ---------------.

There is one scientific issue concerning robots which is sort of interesting.
This is one on which Doug Hofstader may have something to say tomorrow.  If
he hasn't changed his mind, then it is one on which we don't agree.  That is
if you have robots which are intelligent on human level or other computer
programs which are intelligent on human level, would they have human emotions,
or must they?  My view of it is that that would be extra work.  That is, if
you could design such things that would have human emotions, you would have to
solve an additional problem and you would have to do it deliberately, and it
wouldn't be a necessary concomitant of artificial intelligence.  It seems to
me that human emotions have the following characteristics which it would
not be to our advantage to put into robots.  One of them is that subgoals get away
from main goals.  According to one very common theory of morality, people
adopt moral principles in order to please their parents, but then of course
they will adhere to them in opposition to their parents very often.  So we
shouldn't really like that a robot decides to do D as a subgoal of doing A,
that it would then do B in opposition to A.  That's one aspect that we would have 
to build in.  The other aspect which some people have imagined to be automatic
in machines is that our long-term goals are affected by our chemical state.  
When one is in certain kinds of chemical states, nothing seems worthwhile, and
in other states just about anything seems worthwhile.  I don't see that we would
want to make robots that would have that characteristic.  Maybe Doug will say
that we can't help it.

Now I want to go back to reality, that is, go back to things that are in the
immediate future.  What evidence is there as yet for this image of the future
which I have given and in which people have access to all information?  What
evidence is there that people would take to this, that they would buy it?
Of course there isn't much.  The things that people do take to, quite clearly,
are things like message systems.  Even people who have nothing to do with
computers take to message systems.  Word processing has certainly become very
popular even to the extent of people doing their own typography.  It has become
popular around our place and many other places.  The passing around of papers 
through computer has acquired a certain popularity even though there is no decent
way of printing them out.  If we take a Stanford paper and ship it over to the
computer at MIT, they can't print it out in something that has all the nice
italics, etc.  They would surely prefer paper copies.  That is the problem of
standardization.  Certainly it turns out that non-scientists take to it as much
as scientists do, and that typing is not the barrier that some people say it
would be.

Now I want to mention some problems.  The first problem is how we get the
Library of Congress into the computer.  That is going to cost a fair amount of
money whether it's done by retyping it or done by character recognition.  It
turns out that character recognition is one of these scientific problems where
a victory was declared twenty years ago but there is a problem right now that
there is still no company that will take fixed price contracts for reading
books into a computer.

Now there is another social problem which I see, and that is unauthorized
copying.  When you read something, you can short stop it in your terminal,
put it on a file in your terminal, and then give it to your friends without
their having to go to the original file. Then whatever charging mechanism
there is subverted.  I thought about this some time, and I think it would
be quite destructive, but what it could ultimately force is the situation
where almost all writing has to be subsidized, like science is subsidized.
We don't sell scientific papers and put them in the public domain and are paid
for it by the public.  I think that it would be unfortunate for many kinds of
writing.  But there is also the possibility that maybe it won't be so bad. 
Maybe the costs will be low enough that people won't be overly tempted to copy.
Ordinary law enforcement will prevent people from making a business out of it.
It won't be very feasible to make a copy of something and resell it.

Another problem is --------- monopolies.  There are now many database systems
which one can imagine as being precursors of a grand Library of Congress.
There was an article in SCIENCE recently which surveyed them.  They charge
anywhere from $25 an hour to $200  an hour to run up this database, to
massage it, to look at it.  That's a long way from the $1 an hour which I see as
being the feasible cost.  They are already beginning to sue each other about
infringing upon each other's monopolies.  I believe there is a case where
the outfit publishing court decisions in paper form is suing the outfit which is 
presenting the court decisions in computer form.  So something needs to be done
about that.  It is an interesting question of what are the natural monopolies
in this area.  I don't see that there are very many natural monopolies.  One of them
is the one that we've already got, namely, AT & T, that, is, the bare transmission.
Running wires into peoples' homes is a natural monopoly.  There isn't any
natural monopoly in keeping databases.

The last problem that I would mention is the one of fighting off the
technology assessors.  They want to say we'll put it off until we figure it out.
My belief is that they never really will figure it out.  They will not be able
to take everything into account.  For that I've got my last slogan.  Again it
loses -------- from illegibility, which is "Let it be tried.  If it doesn't
work, we can always do something else."

Thank you very much for your attention.